theoretical limit and practical approximation
Probabilistic Inference with Algebraic Constraints: Theoretical Limits and Practical Approximations
Weighted model integration (WMI) is a framework to perform advanced probabilistic inference on hybrid domains, i.e., on distributions over mixed continuous-discrete random variables and in presence of complex logical and arithmetic constraints. In this work, we advance the WMI framework on both the theoretical and algorithmic side. First, we exactly trace the boundaries of tractability for WMI inference by proving that to be amenable to exact and efficient inference a WMI problem has to posses a tree-shaped structure with logarithmic diameter. While this result deepens our theoretical understanding of WMI it hinders the practical applicability of exact WMI solvers to real-world problems. To overcome this, we propose the first approximate WMI solver that does not resort to sampling, but performs exact inference on one approximate models. Our solution performs message passing in a relaxed problem structure iteratively to recover certain lost dependencies and, as our experiments suggest, is competitive with other SOTA WMI solvers.
Review for NeurIPS paper: Probabilistic Inference with Algebraic Constraints: Theoretical Limits and Practical Approximations
Weaknesses: There are two weaknesses with the paper: 1. I am not sure if the theoretical claims are correct. They seem very strong as only in propositional case, we have FPT for constant treewidth, so the claims need to talk about what makes problem so hard when you add in continuous variable. I looked at the proof and there are several things that trouble me: There is change of representation; subset sum is #P-complete when we are concerned with binary representation otherwise we have pseudo-polynomial time algorithms by dynamic programming. The proofs seem to work on integer representation not binary representation as the treewidth for binary representation is still n.
Review for NeurIPS paper: Probabilistic Inference with Algebraic Constraints: Theoretical Limits and Practical Approximations
Two reviewers rated the paper very highly (8, 9). R3 initially gave a very low rating and expressed concerns about correctness of the intractability result, especially related to the representation of numbers in the reduction from subset sum. This point received significant discussion. The meta-reviewer read the proof and felt it was very clear and agrees with the authors. There is one minor ambiguity that can be resolved: the authors don't specify the representation of the constants that appear in the constraints in the WMI problem instance.
Probabilistic Inference with Algebraic Constraints: Theoretical Limits and Practical Approximations
Weighted model integration (WMI) is a framework to perform advanced probabilistic inference on hybrid domains, i.e., on distributions over mixed continuous-discrete random variables and in presence of complex logical and arithmetic constraints. In this work, we advance the WMI framework on both the theoretical and algorithmic side. First, we exactly trace the boundaries of tractability for WMI inference by proving that to be amenable to exact and efficient inference a WMI problem has to posses a tree-shaped structure with logarithmic diameter. While this result deepens our theoretical understanding of WMI it hinders the practical applicability of exact WMI solvers to real-world problems. To overcome this, we propose the first approximate WMI solver that does not resort to sampling, but performs exact inference on one approximate models.